The present invention relates to testing networked devices, and more particularly to testing Internet of Things (IoT) devices and detecting, reporting, and resolving issues identified by the testing.
Testing IoT devices is very challenging because it includes testing multiple, heterogeneous applications distributed across multiple, heterogeneous devices which work cooperatively for a common goal, rather than testing only a single application in a single device. Other challenges involved in IoT testing include a lack of a common protocol followed by the IoT devices and applications. Furthermore, interactions among the IoT devices and applications is complex, which causes difficulties in identifying bottlenecks.
Most IoT testing happens in real-time and runs for a very long time, as such testing facilitates day-to-day human use cases. For example, real-time field trials are typically used in order to test a smart home. An end user (i.e., tester) behaves as a normal user and interacts with IoT devices in the smart home. This real-time testing enables the IoT devices to be tested in actual contexts, which is mandatory for IoT device testing. The tester interacts with IoT devices in the smart home for several days so that several outlier and/or corner cases occur in real-time. Based on a captured manual report regarding the outlier and/or corner cases, suitable recovery or debugging procedures are followed.